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Abstract 

We develop a framework for linear-programming (LP) decoding of non-binary linear codes over rings. We prove that 
the resulting LP decoder has the 'maximum likelihood certificate' property, and we show that the decoder output is 
the lowest cost pseudocodeword. Equivalence between pseudocodewords of the linear program and pseudocodewords 
of graph covers is proved. LP decoding performance is illustrated for the (11, 6, 5) ternary Golay code with ternary 
PSK modulation over AWGN, and in this case it is shown that the LP decoder performance is comparable to 
codeword-error-rate-optimum hard-decision based decoding. 



1 Introduction 

For high-data-rate communication systems, bandwidth- 
efficient signalling schemes are required which neces- 
sitate the use of higher-order modulation. This may 
be achieved in conjunction with coding by the use 
of non-binary codes whose symbols map directly to 
modulation signals. A study of such codes over rings, 
particularly over the integers modulo 8, for use with 
PSK modulation was performed in [7]. 

Of course, within such a framework it is desirable to 
use state-of-the-art error-correcting codes. Low-density 
parity-check (LDPC) codes have become very popular 
in recent years due to their practical effectiveness under 
message-passing decoding. However, the analysis of 
LDPC codes is a difficult task. One approach was 
proposed in [8], and it is based on the consideration 
of so-called pseudocodewords and their pseudoweights. 
The approach was further explored in [3], [6]. In [1] 
and [2], the decoding of binary LDPC codes using 
Unear-programming decoding was proposed, and the 
connections between linear-programming decoding and 
classical belief propagation decoding were established. 
Recently, pseudocodewords of non-binary codes were 
defined and some bounds on the pseudoweights were 
derived in [4]. 

In this work, we extend the approach in [2] towards 
coded modulation, in particular to codes over rings 
mapped to non-binary modulation signals. As was done 
in [2], we show that the problem of decoding may be 
formulated as a linear-programming (LP) problem for 
the non-binary case. We also show that an appropriate 
relaxation of the LP leads to a solution which has 
the 'maximum likelihood (ML) certificate' property. 



i.e. if the LP outputs a codeword, then it must be 
the ML codeword. Moreover, we show that if the LP 
output is integral, then it must correspond to the ML 
codeword. We define the graph-cover pseudocodewords 
of the code, and the LP pseudocodewords of the code, 
and prove the equivalence of these two concepts. This 
shows that the links between LP decoding on the 
relaxed polytope and message-passing decoding on the 
Tanner graph generalize to the non-binary case. 

To demonstrate performance, LP decoding of the 
ternary Golay code is simulated, and the LP de- 
coder is seen to perform approximately as well as 
codeword-error-rate optimum hard-decision decoding, 
and approximately 1.5 dB from the union bound for 
codeword-error-rate optimum soft-decision decoding. 

2 General Settings 

We consider codes over finite rings (this includes codes 
over finite fields, but may be more general). Denote by 
*H a ring with q elements, by its additive identity, and 
let 9^" = 5^\{0}. Let C be a linear [n, k] code with 
parity-check matrix Ti, over [H. The parity check matrix 
Ti has m > n — k rows. 

Denote the set of column indices and the set of 
row indices of by Z = {1,2,-- - ,n} and J = 
{1,2,- •■ , m}, respectively. We use notation Tij for 
the j-th row of TL. Let the graph Q — (V, £) be the 
Tanner graph of C associated with the matrix Ti, namely 
V = {ui,U2, ■ ■ ■ ,M„} U {vi,V2, ■ ■ ■ , Wm}, and there is 
an edge between ui and Vj if and only if Tij i ^ 0. We 
denote by JV{vj) the set of neighbors of the vertex 
Vj, and by supp(c) the support of a vector c. Let 
d = maxjgj{|supp(-Hj)|}. 
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For a word c = (ci, C2, • • • , c„) S ^K", we associate 
the value Cj with variable vertex Uj for each i G I. 
Parity-check j e ^7 is said to be satisfied if and only 
if J2i£x'^j,i • Cj = 0. We say that the vector c is a 
codeword of the single parity-check code Cj if and only 
if parity check j G J is satisfied. Also, we say that the 
vector c is a codeword of C if and only if aU parity 
checks J € J" are satisfied. 

Definition 2.1: ([5]) A graph Q = (V, £) is a finite 
cover of the graph Q = (V, £) if there exists a mapping 
n : V — > V which is a graph homomorphism (11 takes 
adjacent vertices of Q to adjacent vertices of G), such 
that for every vertex v ^ Q and every v E 11^^ (v), 
the neighborhood Af{v) of v is mapped bijectively to 
Af{v). 

Definition 2.2: ([5]) A cover of the graph Q is 
called an M-cover, where M is a positive integer, if 
|n~^(u)| = M for every vertex v €V. 

Fix some positive integer M. Let Q = {V,£) be an 
M-cover of the graph Q = (V, £) representing the code 
C with parity-check matrix Ti. Denote the vertices in the 
sets n~^(ui) and I\r'^{vj) by Wj,2, • ' ' ,Wi,M} 

and {vj^\,Vj^2-, - • ■ ,Vj,M}, respectively, where i G 1 
and j € J. 

Consider the linear code C of length Mn over 91, 
defined by the Mm x Mn parity-check matrix H. For 
1 < < M and i e J, i e J, we let i' = 

{i - 1)M -I- = {j - l)M + j*, and 



n 



3 X 



n 



3,1 





if Ui^i* G J^ivjj* ) 
otherwise 



Then, any vector p G C has the form 

P = {P1,1,P1,2,- ■ ■ ,PlM,P2,l,P2,2, 

■ • ■ ,P2,M, • • • ,Pn,l,Pn,2, ' ' ' iPu^m) ■ 

We associate the value Pi^e, G 9^ with the vertex Ui^e, in 
g (i G J, ^ = 1,2,--- ,M). 

The word p G C as above is called a graph-cover 
pseudocodeword of the code C. Sometimes, we consider 
the following n x q matrix representation, denoted V, 
of the pseudocodeword p: 

where 

mi{a) = \{i& {1,2,- ■■ ,M} : Pi,e = a}\>0, 
for i G X, a G 9^. 

3 Decoding as a Linear- 
Programming Problem 

Assume throughout that the codeword c = 
(ci, C2, • • • , c„) G C has been transmitted over a 
g-ary input memoryless channel, and a corrupted word 
y = (yii 2/2, ■ • • , Vn) G 5]" has been received. Here S 
denotes the set of channel output symbols; we assume 



that this set either has finite cardinality, or is equal 
to MJ or for some integer / > 1. In practice, this 
channel may represent the combination of modulator 
and physical channel. We assume hereafter that all 
information words are equally probable, and so all 
codewords are transmitted with equal probability. 

It was suggested in [1] to represent each symbol as 
a binary vector of length |9l~ |, where the entries in the 
vector are indicators of a symbol taking on a particular 
value. Below, we elaborate on this approach. It should 
be mentioned that by using such a representation, 
the non-binary code is converted into a binary code. 
However, this binary code is not linear, and therefore 
the analysis in [1], [2] is not directly applicable. 

For use in the following derivation, we shall define 
the mapping 



defined by 



91 



{0,1} 



q-1 



c 



X 



{X 



{ay 



'ae<H- 



such that, for all a G 91 



if 6 = a 
otherwise 



We note that the mapping ^(•) is one-to-one, and its 
image is the set of binary vectors of length q — 1 with 
Hamming weight or 1. 

We also define a function A : S — > R U {±00} by 

where, for each y G E, a G 9l~, 



log (eMI] 

Vp(y|a)/ 



and p{y\c) denotes the channel output probability (den- 
sity) conditioned on the channel input. Extend A to a 
map on S" by A(y) = (A(j/i) | A(2/2) | • ■ • | A(y„)). 

The codeword-error-rate-optimum receiver operates 
according to the maximum a posteriori (MAP) decision 
rule: 

c = argmaxp(c|y) 

p{y\c)p{c) 
= argmax— — . 

C6C p{ y ) 

Here p (•) denotes probability if S has finite cardinahty, 
and probabiUty density if E has infinite cardinality. 

By assumption, the a priori probability p{c) is 
uniform over codewords, and p{y) is independent of 
c. Therefore, the decision rule reduces to maximum 
likelihood (ML) decoding: 

c = argmaxp(y|c) 



argmaxTTp(2/^|c,) 



i=l 



= arg max ^ log{p{yi \ci)) 



i=l 
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= argmin V A(2/,;)f(c,)^ , 

cGC ^ — ' 

i=l 

where we have made use of the memoryless property 
of the channel, and of the fact that if Ci = a G 1H~, 
then X{yi)(_{ci)'^ — X^°'\yi). This is then equivalent to 

ie(c2) I ... le(^n)) 

n 

^^''^fflfo^^^y^^f^ (1) 



where 



and 



/ = (/l I /2 



f^ = ift^)c.em- foralHGZ, 

and where /C(C) represents the convex hull of all points 
/ G R(9~i)" which correspond to codewords, i.e. 

IC{C) = i/conv{(e(ci) I e(c2) I . . . I e(c„)) : ceC} . 

Therefore it is seen that the ML decoding problem 
reduces to the minimization of a linear objective func- 
tion (or cost function) over a polytope in M^^"^)". 
The number of variables and constraints for this linear 
program is exponential in n, and it is therefore too 
complex for practical implementation. To circumvent 
this problem, we formulate a relaxed LP problem, as 
shown next. 

The solution we seek for / (i.e. the desired LP 
output) is 



fn) 



f = (e(c-i) I e(c2) 



We introduce auxiliary variables whose constraints, 
along with those of the elements of /, will form the 
relaxed LP problem. First, for each j G J', we define 
the mapping Xj{c) of the words c G W\ Xj{c) = 
(^j,«(c))„gfR-, where 



^].a{c) = e SUpp(Hj) 



for a G Dl^. For each word c G IH", Xj,a{c) is the 
set of word indices where symbol a appears in parity 
check j, for j ^ J , a £ . We define the set Ej as 



In other words, Xj (c) G Ej if and only if parity check 
i is satisfied by the word c G 9^". 

We now introduce the auxiliary variables 

Wj.s for j e J,S e Ej , 

and denote the vector containing these variables as 



with respect to some ordering on the elements of Ej. 
The solution we seek for these variables is 

w ^ r 1 if S^X.j{c) 

1^ otherwise 

To this end, we impose the constraints 

yj e J, \/S e Ej, 0<Wj^s<1, (2) 

and 



(3) 



Finally, we note that the solution we seek satisfies 
the further constraints 



Vj € J,yi€ mppiHj), Va G 



(a) 



(4) 



Constraints ©-(jUi form a polytope which we denote 
Q. The minimization of the objective function ([T]l over 
Q forms the relaxed LP decoding problem. This LP is 
defined by 0{qn + q'^m) variables and 0{qn + q'^m) 
constraints. We note that the further constraints 



and 



ViGX, VaG$K", 0</,^"^<l, (5) 
VzGX, J2 /i"^<l' (6) 



follow from the constraints (|2]i-(|4|i, for any (/, w) G Q. 

Now we may define the decoding algorithm, which 
works as follows. The decoder solves the LP problem 
of minimizing the objective function ^ subject to the 
constraints ©-(gll. If / G {0, the output is the 

codeword (r'(/i), r'(/2), ' • • , T '(/„)) (we shall 
prove in the next section that this output is indeed a 
codeword). Otherwise, the decoder outputs an 'error'. 



4 Polytope Properties 

The analysis in this section is a direct generaUzation of 
the results in [2]. 

Definition 4.1: An integral point in a polytope is 
a point with all integer coordinates. 

Proposition 4.1: 

1) Let {f,w) G Q, and fl"-^ G {0, 1} for every i G 
Z, a G fR". Then, 

(r'(/i),r'(/2),-- - ,r'(/j)ec. 

2) Conversely, for every codeword c = 
(ci, C2, • • ■ , c„) G C, there exists w such 
that {f,w) is an integral point in Q with 
/, = e(c<) for all I G I. 

Proof. 

1) Suppose (/, w) G Q, and /f"^ G {0, 1} for every 
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Define c by q = ^"^(/J for all i G J. By ©, 
this is well defined. Define T = {Ta)aeVi- — 
Xj(c), i.e. 

T„ - e supp(7^j) : /f") = 1} , (7) 

for a e $H^. Now, fix some j ^ J and let P — 
(^Q)aeiR- G Ej, P ^ T. There must exist a G 
*H~ and io G X such that either io G Pa\Ta or 

«0 G Ta\Pa- 

If io e -Pa\T„, then by © and (Hi 



f-^"' = = 

■I In 



Therefore Wj.s = for all S G i?j with io G iSq, 

and in particular Wj,p — 0. 

If io e To\Pa, then by ©, Q, and © 







1 - f .^"^ 



seEj seEj,iaeSa, 

Therefore Wj^s — for all S G Ej with «o ^ Sa, 
and in particular ifj.p ~ 0. 
It follows that 7«j,s = for all S e Ej, S y^T. 
But by Q this implies that T G E'j (and that 
Wj^T = !)■ Applying this argument for every j G 
J' implies c G C 
2) For c G C, we let = £,{ci) for i E T. For each 
parity check j G JT, we let T = {Ta)ae^K- = 
Xj(c) G and then set 

w ^ r 1 ifS' = T 

^ ^ ^^'^ = I otherwise. 

It is easily checked that the resulting point (/, w) 
is integral and satisfies constraints ©-(I?]). □ 

The following proposition assures the so-called ML 
certificate property. 

Proposition 4.2: Suppose that the decoder outputs 
a codeword c G C. Then, c is the maximum-likelihood 
codeword. 

The proof of this proposition is straightforward. The 
reader can refer to a similar proof for the binary case 
in [2]. 

5 Transmission-Independent 
Decoder Performance 

In this section, we state a theorem on decoder perfor- 
mance, namely, that under a certain symmetry condi- 
tion, the probability of decoder failure is independent 
of the transmitted codeword. Decoder failure is defined 
as the event where the decoder output is not equal 
to the transmitted codeword (this could correspond to 
a non-integral value of /, or to an erroneous output 
codeword). 



Symmetry Condition. 

For each a G 9^, there exists a bijection 



such that the channel output probability (density) con- 
ditioned on the channel input satisfies 

p{y\l3) ^ p{Ta{y)\f3 ~ a) , 

For all ?/ G S, /? G 9^. When S is equal to M' or C' for 
^ > 1, the mapping Tq is assumed to be isometric with 
respect to Euclidean distance in E, for every a G 9^. 

Theorem 5.1: Under the stated symmetry condi- 
tion, the probability of decoder failure is independent 
of the transmitted codeword. 

The proof of this theorem is omitted due to space 
limitations. Examples of modulator-channel combina- 
tions for which this assumption holds are: q-ary PSK 
modulation over AWGN (where the additive group of 
^ is cyclic); orthogonal modulation over AWGN; and 
the discrete memoryless q-ary symmetric channel. 

6 Linear-Programming Pseudo- 
codewords 

Definition 6.1: A linear-programming pseudo- 
codeword (LP pseudocodeword) of the code C is a 
vector {h,z) where 

h= {hi\h2\ ■■■ \hn) , 

\/i el, hi = {hi{a))a(,^- , 

where the elements of z are nonnegative integers, and 
the following two conditions hold for all j G J^: 



Vi G supp(Hj), Va G 91", 



(8) 



VzGsupp(7^j), /i,(0)= Y ^i-s- (9) 

SeEj 
VqGK- : ifSa 

From ^ and ^ it follows that the elements of h are 
nonnegative integers, and that for each i G supp(7ij ) n 
supp(Hj'), we have 

aem. SeEj SeEj, 

We assume that the Tanner graph of H is connected; it 
then follows from ( fTol i that 



Vi G X : Y = ^ 



aem 



for some fixed nonnegative integer M. 

We note that the LP pseudocodeword {h,z) defined 
above can be represented by the n x q matrix 



H = (h,ia)) 



el; aeSH 
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In the following, we say that the decoder /ai/s if the 
decoder output is not equal to the transmitted codeword. 

Theorem 6.1: Assume that the all-zero codeword 
was transmitted. 

1) If the LP decoder fails, then there exists some LP 
pseudocodeword (h, z), h 0, such that 

E( E ^^"\y^)h^{a)] <0. (11) 
i=l \ae^- / 

2) If there exists some LP pseudocodeword {h, z), 
h 0, such that 

E( E ^^"HyMa)] <0, (12) 

then the LP decoder fails. 
Proof. The proof follows the lines of its counterpart 
in [2]. 

1) Let be the point in Q which minimizes 
A(y)/^. Suppose the decoder fails; then f ^ 0, 
and we must have \{y)f^ < 0. 

Next, we construct the LP pseudocodeword 
{h, z) as follows. Since the LP has rational 
coefficients, all elements of the vectors / and 
w must be rational. Let M denote their lowest 
common denominator; since / we may have 
M > 0. Now set hi{a) = M ■ /f"^ for all i e I, 
a G 91^, set Zj^s — M ■ Wj^s for all j ^ J 
and S G Ej, and then define hi{Q) as in (|9]l 
for all i G X. By (|2ll and dUl, [h, z) is an LP 
pseudocodeword and h since f ^ 0. Also 
My)f^ < implies (dD- 

2) Now, suppose that an LP pseudocodeword {h, z) 
with h satisfies (fTST i. Let 

A/ = E ^*(") ■ 

Since h / we have M > 0. Now: 

. Set /f"^ = hi{a)/M for alH G X, a G 91"; 
* Set Wj s = Zj s/M for all j e J and S G 

It is straightforward to check that (/, w) satisfies 
all the constraints of the polytope Q. Also, h 
implies / 7^ 0. Finally, (fTST i implies \{y)f'^ < 
0. Therefore, the LP decoder will produce an 
output other than the all-zero codeword, resulting 
in decoder failure. 

□ 

7 Equivalence Between Pseudo- 
codeword Sets 

In this section, we show the equivalence between the 
set of LP pseudocodewords and the set of graph-cover 



pseudocodewords. The result is summarized in the 
following theorem. 

Theorem 7.1: There exists an LP pseudocodeword 
{h, z) for the code C with matrix representation H if 
and only if there exists a graph-cover pseudocodeword 
p with the same matrix representation. 
Proof. 

1) Let {h^ z) be an LP pseudocodeword, and let Q = 
(V, be the Tanner graph C associated with the 
parity-check matrix TL. We define 

M = E ■ 

(Recall that under our assumption that the Tanner 
graph is connected, the value of M is independent 
of i.) Below, we construct a corresponding M- 
cover graph Q = (V, £). 

< For every i G X, and for every a G 9^, 
the graph Q will contain hi{a) copies of the 
vertex associated with the value a. 

• For every j E J , S E Ej, the graph Q will 
contain g copies of the check vertex Vj, 
associated with the {q — l)-tuple S. 

< The edges in the graph are connected accord- 
ing to the membership in the sets Sa, for 
a G y\r . Namely, each copy of check vertex 
Vj will be connected to one copy of Ui for 
every ut G N{vj). A copy of a check vertex 
Vj associated with the ((7 — l)-tuple S will be 
connected to a copy of Ui associated with the 
value a G fH^ if and only if i G S'q. A copy 
of Vj associated with the [q — 1) -tuple S will 
be connected to a copy of Ui associated with 
the value if and only if i ^ yJaern-Sa- 

By using ([8]l, we see that for every j G J, 
i G supp(7ij), a G there are exactly hi{a) 
edges connecting the copies of the vertex Vj with 
the copies of Ui associated with the value a. 
Therefore, the graph Q is well-defined, and the 
neighborhood of a copy of Vj contains exactly 
one copy of Ui for every Ui G N{vj). Further- 
more, it can be seen that the neighborhood of a 
copy of Ui contains exactly one copy of Vj for 
every Vj G N{ui). In addition, all copies of all 
check vertices Vj represent satisfied checks, and 
therefore p, induced by the graph Q, is a graph 
cover pseudocodeword of C, as claimed. 

2) Now let p be a graph-cover pseudocodeword 
corresponding to some ilf -cover of the Tanner 
graph of C. Then, 

• for every i G X, and for every a G we 
define hi{a) to be the number of copies of 
the vertex Ui associated with value a. 

• for every j G J, and for every S G Ej, we 
define Zj s to be the number of copies of the 
check vertex Vj connected to copies of Ui, 
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associated with a £ for i £ Sa, and 
associated with for i ^ Uaem-^a- 

Then, zj^s are all nonnegative integers for all j G 
J and S e Ej. Moreover, (O and ^ hold for all 
j G hy construction of the graph. Therefore, 
{h, z) is an LP pseudocodeword of the code C. 

□ 



8 Simulation Study 

In this section we compare performance of the linear- 
programming decoder with hard-decision and soft- 
decision based ML decoding. For such a comparison, a 
code and modulation scheme are needed which possess 
sufficient symmetry properties to enable derivation of 
analytical ML performance results. We consider en- 
coding of 6-symbol blocks according to the (11,6,5) 
ternary Golay code, and modulation of the resulting 
ternary symbols with 3-PSK modulation prior to trans- 
mission over the AWGN channel. The symbol error 
rate (SER) and codeword error rate (WER) are shown 
in Figure [T] To quantify performance, we define the 
signal-to-noise ratio (SNR) per information symbol 
7s = Es/Nq as the ratio of receive signal energy per 
information symbol to the noise power spectral density. 
Also shown in the figure are two other performance 
curves for WER. The first is the exact result for ML 
hard-decision decoding of the ternary Golay code; since 
the Golay code is perfect, this is obtained from 



11 



WER(7,) = J2 



£=3 
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where p{'~fs) represents the probability of incorrect hard 
decision at the demodulator and was evaluated for each 
value of 7s using numerical integration. The second 
WER curve represents the union bound for ML soft- 
decision decoding. Using the symmetry of the 3-PSK 
constellation, this may be obtained from 



WER(7s) < 



cec \ 



wh{c) r7s 



where r denotes the code rate, and the Hamming weight 
of the codeword c e C, wh{c), is given by the weight 
enumerating polynomial 



W{x) = l + 132x^ + 132a;S + 330x* + 110x^ + 24a;" . 

The performance of LP decoding is approximately the 
same as that of codeword-error-rate optimum hard- 
decision decoding. The performance lies 0.1 dB from 
the result for ML hard-decision decoding and 1.53 dB 
from the union bound for codeword-error-rate optimum 
soft-decision decoding at a WER of 10~^. 
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Fig. L Codeword error rate (WER) and symbol error rate (SER) 
for the (11,6,5) ternary Golay code under 3-PSK modulation. The 
figure shows performance under LP decoding, as well as the exact 
result for hard-decision decoding and the union bound for soft- 
decision decoding. 
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